String Transformation Learning

String t ransformat ion systems have been introduced in (Brill, 1995) and have several applications in natural language processing. In this work we consider the computat ional problem of automatical ly learning from a given corpus the set of transformat ions presenting the best evidence. We introduce an original da ta structure and efficient algori thms that learn some families of t ransformations that are relevant for part-of-speech tagging and phonological rule systems. We also show that the same learning problem becomes NP-hard in cases of an unbounded use of don ' t care symbols in a t ransformation. 1 I n t r o d u c t i o n Ordered sequences of rewriting rules are used in several applications in natural language processing, including phonological and morphological systems (Kaplan and Kay, 1994), morphological disambiguation, part-of-speech tagging and shallow syntactic parsing (Brill, 1995), (Karlsson et ah, 1995). In (Brill, 1995) a learning paradigm, called errordriven learning, has been introduced for automat ic induction of a specific kind of rewriting rules called transformations, and it has been shown that the achieved accuracy of the resulting transformation systems is competi t ive with tha t of existing systems. In this work we further elaborate on the errordriven learning paradigm. Our main contribution is summarized in what follows. We consider some families of t ransformations and design efficient algori thms for the associated learning problem that improve existing methods. Our results are achieved by exploiting a da ta structure originally introduced in this work. This allows us to simultaneously represent and test the search space of all possible transformations. The t ransformations we investigate make use of classes of symbols, in order to generalize regularities in rule applications. We also show that when an unbounded number of these symbol classes are allowed within a transformation, then the associated learning problem becomes NP-hard. The notat ion we use in the remainder of the paper is briefly introduced here. ~3 denotes a fixed, finite alphabet and e the null string. E* and E+ are the set of all strings and all non-null strings over E, respectively. Let w 6 E*. We denote by Iwl the length o f w . Let w = uxv; u i s a p r e f i x and v is a suffix of w; when x is non-null, it is called a f a c t o r of w. The suffix of w of length i is denoted suf f i (w) , for O < i _< Iwl. Assume that x is non-null, and w = u ixsu f f i (w ) for ~ > 0 different values of i but not for ~ + 1, or x is not a factor of w and ~ = 0. Then we say that ~ is the statistic of factor z in w. 2 T h e l e a r n i n g p a r a d i g m The learning paradigm we adopt is called errordriven learning and has been originally proposed in (Brill, 1995) for part of speech tagging applications. We briefly introduce here the basic assumptions of the approach. A string t r a n s f o r m a t i o n is a rewriting rule denoted as u -* v, where u and v are strings such that [u[ = Ivt. This means that i fu appears as a factor of some string w, then u should be replaced by v in w. The application of the t ransformation might be conditioned by the requirement tha t some additionally specified pattern matches some part of the string w to be rewritten. We now describe how transformations can be automatically learned. A pair of strings (w, w ' ) is an a l i g n e d p a i r if IT[ = ]w'[. When w = uzsu f f i (w) , w' = u ' x ' su f f i (w ' ) and Ixl = Ix'l, we say that factors x and x' occur at aligned positions within (w, w'). A multi-set of aligned pairs is called an a l i g n e d c o r p u s . Let (w, w ') be an aligned pair and let 7be some transformation of the form u --~ v. The p o s i t i v e e v i d e n c e of v (w.r.t. (w, w')) is the number of different positions at which factors u and v are aligned within (w, w'). The n e g a t i v e ev i d e n c e of r (w.r.t. w, w ~) is the number of different positions at which factors u and u are aligned within